In a coded-light three-dimensional (3D) camera, such as Intel® RealSense®, the scene is illuminated by a sequence of binary code patterns (e.g., vertical stripes of different width), which are then captured by the camera. The resulting sequence of images captured by the camera undergoes binarization, producing in each pixel seen by the camera a binary code representing a one-dimensional (1D) coordinate in a projector system of coordinates; using the code, it is possible to infer the depth of the scene by triangulation from the known relative positions of the camera and a projector. The accuracy of code estimation depends on the quality of the underlying binarization process, which may vary at each pixel. Binarization process provides a level of confidence, which can be used as an indicator of the confidence of the code estimation in a pixel.
Another aspect of discussion is code discontinuities, such as the points in which the code transitions from 0 to 1 or vice versa. In principle, these transitions contain all the information about the depth of the scene, and thus, ideally, it is desirable to determine the code transition locations with sub-pixel precision.
It is contemplated that the quality of the code accuracy can have a dramatic effect on the resulting depth image quality since the triangulation process amplifies errors. Typically, errors in the code image are translated into saw-tooth-like artifacts in the final depth image. Therefore, it is desired to obtain the highest possible code image.
Furthermore, imaging moving scenes (in which some objects are moving) may result in displacement of code patterns with respect to each other, leading to inconsistent code values.
It is desirable to provide a way to improve the quality of the code image taking into account the code estimation confidence, and in particular, pixels in which code transitions are estimated with higher precision. Simple filtering of the code image by conventional filters (e.g., bilateral filter) does not account for special structure of the code and may only increase artifacts instead of reducing them.